Quarto, RStudio and Reproducibility

Pablo Mosteiro

Methodology & Statistics @ UU University

Hanne Oberman

Methodology & Statistics @ UU

16 Jan 2026

Disclaimer

These materials are generated by Gerko Vink, who holds the copyright. He thanks many people, as the thoughts and code in these slides are the result of years-long development cycles. When someone has contributed to the content of the slides, he has credited their authorship.

The intellectual property belongs to Utrecht University. Images are either directly linked, or generated with StableDiffusion or DALL-E. There is no information here that exceeds legal use of copyright materials in academic settings, or that should not be part of the public domain.

Warning

You may use any and all content in this presentation - excluding names - and submit it as input to generative AI tools, with the following exception:

  • You must ensure that the content is not used for further training of the model

Source:

Disclaimer #2

We have limited time and so much to talk about. Today we therefore focus on creating reproducible research compendiums with Quarto and developing R packages with RStudio.

We will not focus on programming. After all, programming you learn by doing, not by listening. If you’d like to learn more about programming, I suggest you take a look at the following resources: https://www.gerkovink.com/rijkR/. This is a different version of this course, but it contains a lot of useful information.

That said, let’s at least cover the programming basics.

R Programming basics

  • classes in R
  • vectors, matrices, arrays, data frames, tibbles
  • pipes
    • |>, %>%, %$%, %T>%, %<>%.
  • future
  • purrr and the map/Reduce workflows
  • ggplot2 and plotly
  • targets (formerly drake)
  • rng’s and set.seed()

This lecture

  • Reproducibility
  • Research compendiums
  • Quarto

Why bother?

We would like our results to be as fully reproducible as possible:

A. Reproducibility is one of the pillars of science

  • It is the standard by which to judge scientific claims
  • It helps the cumulative growth of knowledge - no duplication of effort

B. Reproducibility may greatly benefit you

  • You’ll develop better work habits
  • Better teamwork - especially with new team members
  • Changing or amending your work is much easier
  • Higher research impact - more likely to be picked up and cited

By the end of today…

  • … you will have heard what reproducible research and programming is and why it is important
  • … you will be able to produce a reproducible research archive from scratch
  • … you’ll have published an open R package with proper unit tests
  • … and much more!

Definition

A result is reproducible when the same analysis steps performed on the same dataset consistently produces the same answer.

Reproducibility of R scripts

Reproducible research is not the norm:

74% of R files failed to complete without error

Research compendium

Definition

  • “A research compendium is a collection of all digital parts of a research project including
    • data,
    • code,
    • texts
  • The collection is created in such a way that reproducing all results is straightforward”1
  • “The compendium serves as a means for distributing, managing, and updating the collection”2

Basic compendium

A basic research compendium is just a folder…

compendium/
├── data
│   └── my_data.csv
├── analysis
│   └── my_script.R
├── requirements.txt
└── README.md

(Not so) basic compendium

… but it can become extensive…

|
├── paper/
│   ├── paper.qmd       
│   └── references.bib  
| 
├── figures/            
|
├── data/
│   ├── raw_data/       
│   └── clean_data/   
|
└── templates
    └── journal_template.csl     

(Not so) basic compendium

…or even executable!

|
├── _targets.R
├── R/
│   ├── functions_data.R
│   ├── functions_analysis.R
│   ├── functions_visualization.R
└── data/
    └── input_data.csv

RStudio

RStudio - Integrated Development Environment (IDE)

  • RStudio is a powerful IDE designed specifically for R.
  • It offers a user-friendly interface and a suite of tools to enhance your R coding experience.

Key Features of RStudio

  • 1. Code Editing: syntax highlighting, auto-completion, and error checking.

  • 2. Console: allows you to execute R code line by line and view results in real time.

  • 3. Environment Pane: Keep track of your variables, data frames, and functions.

  • 4. Plots and Visualizations: Create and view plots, charts, and visualizations within RStudio.

  • 5. Integrated Help: Access R documentation, packages, and online resources.

  • 6. Version Control: Easily integrate R projects with version control systems like Git.

  • 7. Markdown Support: seamlessly integrated, making RStudio an ideal choice for creating reproducible reports and documents.

It plays a crucial role in promoting reproducibility and collaboration in data science and statistical analysis.

RMarkdown, GitHub-Flavored Markdown, and Quarto

  • Markdown

    • lightweight markup language for creating formatted text using plain text
    • easy to learn and widely used in various applications.
  • GitHub-Flavored Markdown (GFM)

    • variant of Markdown used on GitHub.
  • RMarkdown

    • extension of Markdown that allows you to embed R code and its output directly within a document.
    • Perfect for creating reproducible reports, research papers, and presentations.
  • Quarto

    • comprehensive tool for creating reproducible and collaborative data science documents.
    • Combines Markdown, \(\LaTeX\), and code execution to streamline document preparation.

Structure of a Quarto Document

  • Organized into blocks, the fundamental building components of the document’s content.
  • Each block can contain a combination of the following elements:
    • Text: Narrative content and explanations.
    • Code: R code chunks for computations and data analysis.
    • Equations: Mathematical equations and notation.
    • Figures: Visualizations, charts, and images.
    • Tables: Data tables for presenting results.
    • YAML Headers: Metadata and document configuration.
    • Sections and Subsections: Organizing the document into a hierarchy.

Quarto: the YAML header

YAML (YAML Ain’t Markup Language)

  • In Quarto and many other applications, YAML is used to specify:

    • Document Metadata: title, author, date, document type, etc

    • Document Configuration: output format (e.g., HTML, PDF), document template, style options, etc

    • Custom Variables: variables/parameters that can be used throughout the document to control behavior or content

Here’s an example of a simple YAML header in a Quarto document:

---
title: "All flavors markdown"
author: 
  - name: Gerko Vink
    orcid: 0000-0001-9767-1924
    email: g.vink@uu.nl
    affiliations:
      - name: Methodology & Statistics @ UU University
  - name: Hanne Oberman
    orcid: 0000-0003-3276-2141
    email: h.i.oberman@uu.nl
    affiliations:
      - name: Methodology & Statistics @ UU
date: 13 Sep 2023
date-format: "D MMM YYYY"
bibliography: data/publications.bib
execute: 
  echo: true
editor: source
format: 
  revealjs:
    theme: [solarized, gerko.scss]
    progress: true
    multiplex: true
    transition: fade
    slide-number: true
    margin: 0.075
    logo: "images/logo.png" 
    toc: false
    toc-depth: 1
    toc-title: Outline
    scrollable: true
    reference-location: margin
    footer: Gerko Vink and Hanne Oberman - Markup Languages @ UU
---

In this example:

  • title, author, and date provide metadata about the document.
  • output specifies settings related to the document’s output format and theme.

The YAML header allows you to control how the document is rendered and presented. It ensures that important document information and settings are stored in a human-readable and structured format at the beginning of the document.

Text in Quarto

Text is text. Nothing more, nothing less

Subheader

Subsubheader
# This is a heading indicating a section
## This is a heading indicating a subsection
### This is a heading indicating a subsubsection

But in the above I used

### Header
#### Subheader
##### Subsubheader


Figures

A surfing elephant

The above figure is left-aligned
The caption is also left-aligned

A surfing elephant

The above figure is centered
The caption is still left-aligned

The code for the figures is

![A surfing elephant](images/lec-2/elephantastic.png){fig-align="left"}
![A surfing elephant](images/lec-2/elephantastic.png){fig-align="center"}

Columns, Sizing, Equations and proper centering

A surfing \(\epsilon\lambda\epsilon\phi \alpha\nu\tau\)

\(e^{i\pi} + 1 = 0\)

The above figure is left-aligned
The caption is also left-aligned
The equation is left-aligned too

A surfing elephant

\[e^{i\pi} + 1 = 0\]

The above figure is centered
The caption is also centered
The equation is centered

The code for the slide is

## Columns, Sizing, Equations and proper centering

::: columns
::: {.column width="50%"}
![A surfing $\epsilon\lambda\epsilon\phi h\alpha\nu\tau$](images/lec-2/elephantasticer.png){fig-align="left" width="80%"}

$e^{i\pi} + 1 = 0$

The above figure is left-aligned <br>
The caption is also left-aligned <br>
The equation is left-aligned too
:::

::: {.column width="50%"}
<center>
![A surfing elephant](images/lec-2/elephantasticer.png){fig-align="center" width="80%"}
</center>

$$e^{i\pi} + 1 = 0$$

The above figure is centered <br>
The caption is also centered <br>
The equation is centered
:::

:::

The code for the slide is

``` markdown
## Columns, Sizing, Equations and proper centering

::: columns
::: {.column width="50%"}
![A surfing $\epsilon\lambda\epsilon\phi h\alpha\nu\tau$](images/lec-2/elephantasticer.png){fig-align="left" width="80%"}

$e^{i\pi} + 1 = 0$

The above figure is left-aligned <br>
The caption is also left-aligned <br>
The equation is left-aligned too
:::

::: {.column width="50%"}
<center>
![A surfing elephant](images/lec-2/elephantasticer.png){fig-align="center" width="80%"}
</center>

$$e^{i\pi} + 1 = 0$$

The above figure is centered <br>
The caption is also centered <br>
The equation is centered
:::

:::

R figures

library(dygraphs)
dygraph(nhtemp, main = "New Haven Temperatures", ylab = "Temp (F)") 
```{r, fig.width=15, fig.height=4}
library(dygraphs)
dygraph(nhtemp, main = "New Haven Temperatures", ylab = "Temp (F)") 
```

R figures

library(dygraphs)
dygraph(nhtemp, main = "New Haven Temperatures", ylab = "Temp (F)") 
```{r, fig.width=6, fig.height=2.5}
library(dygraphs)
dygraph(nhtemp, main = "New Haven Temperatures", ylab = "Temp (F)") 
```

R figures

library(ggplot2, warn.conflicts = FALSE)
library(plotly, warn.conflicts = FALSE)
ggplot(mpg, aes(displ, hwy, colour = class)) +
  geom_point() +
  geom_smooth(se = FALSE, method = lm) 

R figures

library(ggplot2, warn.conflicts = FALSE)
library(plotly, warn.conflicts = FALSE)
p <- ggplot(mpg, aes(displ, hwy, colour = class)) +
  geom_point() +
  geom_smooth(se = FALSE, method = lm) 
p %>% ggplotly()

Equations

  • $\mu$ is used for in-line equations
  • $$\mu$$ is used for equations

Let’s assume that \(y\) is a vector with \(N\) elements such that \[y \sim \mathcal{N}(\mu, \sigma^2),\] where we set in our simulations \(\mu = 10\) and \(\sigma^2 = 5\). We do something for every \(y_i\) with \(i = 1, \dots, N\).

Code chunks

We already saw some markdown code chunks. We can run inline code to calculate that 12 * 234 is 2808 by using `r ` and code chunks to evaluate larger blocks of code, such as:

Code
library(ggplot2)

ggplot(mpg, aes(x = hwy, y = cty, color = cyl)) +
  geom_point(alpha = 0.5, size = 2) +
  scale_color_viridis_c() +
  theme_minimal()

ggplot(mpg, aes(x = hwy, y = cty, color = displ)) +
  geom_point(alpha = 0.5, size = 2) +
  scale_color_viridis_c(option = "E") +
  theme_minimal()
(a) Color by number of cylinders
(b) Color by engine displacement, in liters
Figure 1: City and highway mileage for 38 popular models of cars.

The previous slide

## Code chunks and explicit caching

```{r}
#| output: asis

answer <- 12 * 234
``` 

We already saw some `markdown` code chunks. We can run inline code to calculate 
that `12 * 234` is ` r answer ` by using `r  ` and code chunks to evaluate 
larger blocks of code, such as:

```{{r}}
#| output: asis
#| label: fig-mpg
#| fig-cap: "City and highway mileage for 38 popular models of cars."
#| fig-subcap:
#|   - "Color by number of cylinders"
#|   - "Color by engine displacement, in liters"
#| layout-ncol: 2
#| column: page
#| code-fold: true
library(ggplot2)

ggplot(mpg, aes(x = hwy, y = cty, color = cyl)) +
  geom_point(alpha = 0.5, size = 2) +
  scale_color_viridis_c() +
  theme_minimal()

ggplot(mpg, aes(x = hwy, y = cty, color = displ)) +
  geom_point(alpha = 0.5, size = 2) +
  scale_color_viridis_c(option = "E") +
  theme_minimal()
```

Explicit Caching

  • technique used in RMarkdown / Quarto to reuse the results of expensive steps
  • improves efficiency of document generation and preserves reproducibility

Why Use it?

  • Efficiency: Caching saves time and resources by avoiding repeated execution.

  • Reproducibility: ensures the same results are used when rendering the document, even if the code is modified or re-executed.

  • Version Control: Caching reduces the size of version-controlled documents by storing intermediate results separately.

How Does Explicit Caching Work?

In RMarkdown and Quarto, you can use cache directives to specify which code chunks should be cached and which should not.

  • Cached chunks are identified by a unique hash based on their code content, ensuring consistency.

The following code block

```{r}
#| cache: true
#| code-line-numbers: true
#| label: imputation
library(mice)
library(magrittr)
library(purrr)
imp <- nhanes %>% mice(print = FALSE)
imp |>
  complete("all") %>% 
  map(~.x %$% lm(bmi ~ chl + hyp)) |>
  pool()
```

would result in this output

library(mice)
library(magrittr)
library(purrr)
imp <- nhanes %>% mice(print = FALSE)
imp |>
  complete("all") %>% 
  map(~.x %$% lm(bmi ~ chl + hyp)) |>
  pool()
Class: mipo    m = 5 
         term m    estimate         ubar           b            t dfcom
1 (Intercept) 5 18.78484224 1.668347e+01 1.63902e+00 1.865029e+01    22
2         chl 5  0.04265802 4.069179e-04 8.21193e-05 5.054611e-04    22
3         hyp 5 -0.46648713 4.067300e+00 1.00688e+00 5.275556e+00    22
        df       riv    lambda       fmi
1 17.23778 0.1178906 0.1054581 0.1938612
2 14.10953 0.2421696 0.1949570 0.2890616
3 12.95373 0.2970659 0.2290291 0.3256800

with corresponding cache structure relative to the root and format of the output file:

Tables

Use https://www.tablesgenerator.com/markdown_tables if you need to make a table from scratch.

Otherwise, use tibbles:

mice::boys |> 
  tibble::as_tibble()
# A tibble: 748 × 9
     age   hgt   wgt   bmi    hc gen   phb      tv reg  
   <dbl> <dbl> <dbl> <dbl> <dbl> <ord> <ord> <int> <fct>
 1 0.035  50.1  3.65  14.5  33.7 <NA>  <NA>     NA south
 2 0.038  53.5  3.37  11.8  35   <NA>  <NA>     NA south
 3 0.057  50    3.14  12.6  35.2 <NA>  <NA>     NA south
 4 0.06   54.5  4.27  14.4  36.7 <NA>  <NA>     NA south
 5 0.062  57.5  5.03  15.2  37.3 <NA>  <NA>     NA south
 6 0.068  55.5  4.66  15.1  37   <NA>  <NA>     NA south
 7 0.068  52.5  3.81  13.8  34.9 <NA>  <NA>     NA south
 8 0.071  53    3.89  13.8  35.8 <NA>  <NA>     NA west 
 9 0.071  55.1  3.88  12.8  36.8 <NA>  <NA>     NA west 
10 0.073  54.5  4.2   14.1  38   <NA>  <NA>     NA east 
# ℹ 738 more rows

or use e.g. library(DT) for customization:

library(DT)
mtcars %>% 
  datatable(options = list(pageLength = 25))

reprex and renv

library(renv)
#> 
#> Attaching package: 'renv'
#> The following objects are masked from 'package:stats':
#> 
#>     embed, update
#> The following objects are masked from 'package:utils':
#> 
#>     history, upgrade
#> The following objects are masked from 'package:base':
#> 
#>     autoload, load, remove
renv::init()
#> - Linking packages into the project library ... Done!
#> The following package(s) will be updated in the lockfile:
#> 
#> # CRAN -----------------------------------------------------------------------
#> - backports       [* -> 1.4.1]
#> - base64enc       [* -> 0.1-3]
#> - broom           [* -> 1.0.4]
#> - bslib           [* -> 0.4.2]
#> - cachem          [* -> 1.0.8]
#> - cli             [* -> 3.6.1]
#> - cpp11           [* -> 0.4.3]
#> - digest          [* -> 0.6.31]
#> - dplyr           [* -> 1.1.2]
#> - ellipsis        [* -> 0.3.2]
#> - evaluate        [* -> 0.21]
#> - fansi           [* -> 1.0.4]
#> - fastmap         [* -> 1.1.1]
#> - fontawesome     [* -> 0.5.1]
#> - fs              [* -> 1.6.2]
#> - generics        [* -> 0.1.3]
#> - glue            [* -> 1.6.2]
#> - highr           [* -> 0.10]
#> - htmltools       [* -> 0.5.5]
#> - jquerylib       [* -> 0.1.4]
#> - jsonlite        [* -> 1.8.4]
#> - knitr           [* -> 1.43]
#> - lattice         [* -> 0.21-8]
#> - lifecycle       [* -> 1.0.3]
#> - magrittr        [* -> 2.0.3]
#> - memoise         [* -> 2.0.1]
#> - mime            [* -> 0.12]
#> - pillar          [* -> 1.9.0]
#> - pkgconfig       [* -> 2.0.3]
#> - purrr           [* -> 1.0.1]
#> - R6              [* -> 2.5.1]
#> - rappdirs        [* -> 0.3.3]
#> - Rcpp            [* -> 1.0.11]
#> - RcppArmadillo   [* -> 0.12.4.0.0]
#> - renv            [* -> 1.0.2]
#> - rlang           [* -> 1.1.1]
#> - rmarkdown       [* -> 2.22]
#> - sass            [* -> 0.4.6]
#> - stringi         [* -> 1.7.12]
#> - stringr         [* -> 1.5.0]
#> - tibble          [* -> 3.2.1]
#> - tidyr           [* -> 1.3.0]
#> - tidyselect      [* -> 1.2.0]
#> - tinytex         [* -> 0.45]
#> - utf8            [* -> 1.2.3]
#> - vctrs           [* -> 0.6.2]
#> - withr           [* -> 2.5.0]
#> - xfun            [* -> 0.39]
#> - yaml            [* -> 2.3.7]
#> 
#> # GitHub ---------------------------------------------------------------------
#> - mice            [* -> gerkovink/mice@match_conditional]
#> 
#> The version of R recorded in the lockfile will be updated:
#> - R               [* -> 4.3.0]
#> 
#> - Lockfile written to "/private/var/folders/yx/6rn4qpl13wsgk4c7s3jc9d1r0000gp/T/RtmpMqetyA/reprex-1233f6d959b19-drear-kitty/renv.lock".
library(mice)
#> 
#> Attaching package: 'mice'
#> The following object is masked from 'package:stats':
#> 
#>     filter
#> The following objects are masked from 'package:base':
#> 
#>     cbind, rbind
library(magrittr)
library(purrr)
#> 
#> Attaching package: 'purrr'
#> The following object is masked from 'package:magrittr':
#> 
#>     set_names
#> The following object is masked from 'package:renv':
#> 
#>     modify
renv::snapshot()
#> - The lockfile is already up to date.
renv::remove("mice")
#> - Removing package(s) from project library ...
#> Removing package 'mice' ... Done!
renv::restore()
#> The following package(s) will be updated:
#> 
#> # GitHub ---------------------------------------------------------------------
#> - mice   [* -> gerkovink/mice@match_conditional]
#> 
#> # Installing packages --------------------------------------------------------
#> - Installing mice ...                           OK [linked from cache]
imp <- nhanes %>% mice(print = FALSE)
imp |>
  complete("all") %>% 
  map(~.x %$% lm(bmi ~ chl + hyp)) |>
  pool()
#> Class: mipo    m = 5 
#>          term m   estimate         ubar            b            t dfcom
#> 1 (Intercept) 5 20.6687887 2.020809e+01 8.8963873222 3.088375e+01    22
#> 2         chl 5  0.0354435 5.409979e-04 0.0001081414 6.707676e-04    22
#> 3         hyp 5 -0.8447582 4.353337e+00 0.9275040571 5.466342e+00    22
#>          df       riv    lambda       fmi
#> 1  9.489411 0.5282868 0.3456725 0.4504537
#> 2 14.161181 0.2398710 0.1934645 0.2874598
#> 3 13.811544 0.2556671 0.2036105 0.2983537

sessionInfo()
#> R version 4.3.0 (2023-04-21)
#> Platform: aarch64-apple-darwin20 (64-bit)
#> Running under: macOS Ventura 13.6
#> 
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRblas.0.dylib 
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.11.0
#> 
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> time zone: Europe/Amsterdam
#> tzcode source: internal
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] purrr_1.0.1    magrittr_2.0.3 mice_3.15.3    renv_1.0.2    
#> 
#> loaded via a namespace (and not attached):
#>  [1] vctrs_0.6.2      cli_3.6.1        knitr_1.43       rlang_1.1.1     
#>  [5] xfun_0.39        generics_0.1.3   glue_1.6.2       backports_1.4.1 
#>  [9] htmltools_0.5.5  fansi_1.0.4      rmarkdown_2.22   grid_4.3.0      
#> [13] evaluate_0.21    tibble_3.2.1     fastmap_1.1.1    yaml_2.3.7      
#> [17] lifecycle_1.0.3  compiler_4.3.0   dplyr_1.1.2      fs_1.6.2        
#> [21] pkgconfig_2.0.3  Rcpp_1.0.11      tidyr_1.3.0      rstudioapi_0.14 
#> [25] lattice_0.21-8   digest_0.6.31    R6_2.5.1         tidyselect_1.2.0
#> [29] reprex_2.0.2     utf8_1.2.3       pillar_1.9.0     tools_4.3.0     
#> [33] withr_2.5.0      broom_1.0.4

Created on 2023-09-27 with reprex v2.0.2

Benefits of Using renv

  • 1. Dependency Isolation: renv creates a dedicated environment for each project, preventing conflicts between package versions.

  • 2. Reproducibility: With renv, you can capture and record the specific package versions used in your project, ensuring reproducibility over time.

  • 3. Collaboration: Share your project with others, and they can easily recreate the same environment using renv.lock.

  • 4. Easier Package Installation: renv simplifies package installation. Just run renv::restore() to set up the project environment.

  • 5. Automatic Snapshot: renv automatically generates a renv.lock file, listing all package versions used, making it easy to recreate the environment.

  • 6. Compatibility: renv is compatible with popular version control systems like Git, facilitating collaboration and sharing.

Use renv::snapshot() to update the renv.lock file when you add or update packages. You can use renv::activate() and renv::deactivate() to activate or deactivate renv for your project.

Diagrams

```{{mermaid}}
%%| echo: false
flowchart LR
  A[Hard edge] --> B(Round edge)
  B --> C{Decision}
  C <--> D[Result one]
  C --> E[Result two]
```


flowchart LR
  A[Hard edge] --> B(Round edge)
  B --> C{Decision}
  C <--> D[Result one]
  C --> E[Result two]

See Quarto Diagrams for a more comprehensive overview of all graphing engines. This Live online Mermaid editor is awesome!

Scientific publishing with rticles and Quarto

To use rticles from RStudio, you can access the templates through File -> New File -> R Markdown. This will open the dialog box where you can select from one of the available templates:

Quarto

The quarto use template command can be used to create an article from one the below formats.

Terminal
quarto use template quarto-journals/acm
quarto use template quarto-journals/plos
quarto use template quarto-journals/elsevier
quarto use template quarto-journals/acs
quarto use template quarto-journals/jss

Citations

Oberman and Vink (2023) and Cai, Van Buuren and Vink (2023) are some recent publications. Some references - such as some work by Reinder Banning and Gerko Vink are much older (2010) or contain simple and contemporary solutions by Volker and Vink (2022, 2–4) or cool but potentially confusing images (Schouten and Vink 2021, 1255).

Banning, Reinder, and Gerko Vink. 2010. “Methoden En Indicatoren Voor Het Evalueren En Beschrijven van Controle- En Correctieprocessen.” CBS nota DMV-2010- 06-08-RBNG-GVNK. Centraal Bureau voor de Statistiek.
Cai, Mingyang, Stef van Buuren, and Gerko Vink. 2023. “Graphical and Numerical Diagnostic Tools to Assess Multiple Imputation Models by Posterior Predictive Checking.” Heliyon.
Oberman, Hanne I., and Gerko Vink. 2023. “Toward a Standardized Evaluation of Imputation Methodology.” Biometrical Journal, 2200107. https://doi.org/10.1002/bimj.202200107.
Schouten, Rianne Margaretha, and Gerko Vink. 2021. “The Dance of the Mechanisms: How Observed Information Influences the Validity of Missingness Assumptions.” Sociological Methods & Research 50 (3): 1243–58.
Volker, Thom, and Gerko Vink. 2022. “Synthetic Data in r: Generating Synthetic Data with High Utility Using Mice.” Https://Www.gerkovink.com/miceVignettes/Synthetic/Synthetic.html.


## Citations

@eval and Cai, Van Buuren and Vink [-@cai2023graphical] are recent publications. Some references - such as some work by Reinder Banning and Gerko Vink are much older [-@banningvink] or contain simple and contemporary solutions by @vigntg [p. 2-4] or cool but potentially confusing images [@schouten2021dance, 1255].

::: footer
But you can also refer to @eval here
:::

::: {#refs}
:::